(19) 




Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(11) EP 0 713 335 A2 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

22.05.1996 Bulletin 1996/21 

(21) Application number: 95307885.4 

(22) Date of filing: 06.11.1995 



(51) Intel 6: H04N7/08 



(84) Designated Contracting States: 


• Herbst, Steven M. 


DE ES FR GB 


Bernardsville, New Jersey 07924 (US) 




• Sizer II, Theodore 


(30) Priority: 15.11.1994 US 339538 


Little Silver, New Jersey 07739 (US) 


(71) Applicant: AT&T Corp. 


(74) Representative: 


New York, NY 10013-2412 (US) 


Buckley, Christopher Simon Thirsk et al 


(72) Inventors: 


AT&T (UK) LTD., 


AT&T Intellectual Property Division, 


• August, Katherine G. 


5 Mornington Road 


Matawan, New Jersey 07747 (US) 


Woodford Green, Essex IG8 OTU (GB) 


• Caldwell, Charles D. 


Cliffwood Beach, New Jersey 07735 (US) 





CSI 

< 

CO 
CO 

CO 

hi 
o 

Q. 
UJ 



(54) System and method for wireless capture of encoded data transmitted with a television, video 
or audio signal and subsequent initiation of a transaction using such data 

(57) A device (11 0) which can be portable and there- 
fore used in conjunctbn with several different data 
sources is arranged for wireless capture of encoded 
non -perceptible data directly from a light emitting de- 
vice, such as a televisbn (101) or other video and/or 
audio output device. The data is part of the ordinary pro- 
gram displayed by the device (101 ) and yet indiscernible 
by a viewer or listener. The indiscernible data may be 
information needed to order those services or products, 
including prices, delivery inten^als, shipping details, 
coupon offers, and so-on. After capture, the device in- 
cl udes a memory arranged to store the data for later use, 
and communications circuitry (114 or 118) arranged to 
output at least portions of the data (or data derived from 
the captured data) to a remote agent or device 
(1 20, 1 21 ), such as via a telephone call (via 1 40 or 1 09 
or 112) or a data transfer to a point of sale device. 
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Description 

Field of the Invention 

This Invention relates to a device and a method for s 
the wireless capture of encoded non-perceptible data 
from a light emitting device such as a television or other 
video and/or audio output device, and subsequent use 
of the captured data to initiate a transaction, such as 
placing a telephone call and transmitting a message to 10 
a remote agent or automated attendant, or delivering the 
data to a point of sale agent directly or indirectly via a 
telecommunications connection, or to operate another 
device. 

15 

Background of the Invention 

In a copending application entitled "System and 
Method of Capturing Encoded Data Transmitted Over a 
Communications Network in a Video System" filed on 20 
behalf of Isenberg and Tuomenoksa on March 18,1 994, 
Serial No. 08/21 0802 and assigned to the same assign- 
ee as the present application, it was recognized that 
non-perceptible information can be encoded in a televi- 
sion program, captured in a "set top box", and thereafter 25 
used to make telephone calls. The set top box is an elec- 
trical devbe that is "hard-wired" directly in the circuit be- 
tween the video sen/ices network and the television set 
(as a stand ak>ne unit or alternatively as part of the tel- 
evision set Itself) and thus, unfortunately, Is not portable. 30 
Also, the set top box is dedk:ated to one particular tele- 
vision (or other video source) and thus cannot be shared 
among several sources. Finally, the user of the set top 
box may be uncertain as to the nature of the infomiation 
captured therein. 35 

Summary of the invention 

In accordance with our Invention, a device which 
can be portable and therefore used In conjunction with ^ 
several different data sources is arranged for wireless 
capture of encoded non-perceptible data directly from a 
light emitting device, such as a television or other video 
and/or audb output device. The data is part of the ordi- 
nary program displayed by the device and yet indlscem- 45 
Ible by a viewer or listener. As an example, a televlston 
monitor may be kx:ated In a kk>sk or other publk; area, 
and display a program advertising selected services and 
products. The indiscernible data may be information 
needed to order those services or products, including so 
prices, delivery inten^ls, shipping details, coupon of- 
fers, and so-on. 

Capture can be effected by decoding subtimlnally 
modulated video broadcast signals. In a manner sug- 
gested in U. S. Patent 4,807,031, Issued on February ss 
21, 1989. to R. S. Broughton, et al., the teachings of 
whrch are incorporated herein by reference. Afternative- 
ly, a modulated carrier, which, for example, can be an 
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amplitude shift keyed (ASK) or frequency shift keyed 
(FSK) signal having its primary frequency components 
at a frequency (e.g. 25 kHz) not usually present In a con- 
ventional television signal, can be added only to the lu- 
minance component of a television signal, and the data 
can thereafter be captured using bandpass filtering and 
an ASK or FSK receiver. After capture, the device in- 
cludes a memory arranged to store the data for later use, 
and communications circuitry arranged to output at least 
portions of the data (or data derived from the captured 
data) to a remote agent or device, such as via a tele- 
phone call or a data transfer to a point of sale device. 

In accordance with the present Invention, the en- 
coded non-perceptible data irKludes associated dialing, 
routing, or identification informatk>n, so that other por- 
tions of the captured data can be transmitted to an ap- 
propriate final destination or device. In this context, the 
final destinatbn can be (a) a renrK>te location, such as a 
platform arranged to process and consummate transac- 
tions, or (b) a point of sale system or other devrce (e.g., 
TV, stereo, VCR Plus, smart mouse device, etc.). The 
other portions of the captured data that constitutes a 
payload to be communicated to the final destination or 
device, can, for example, be product or coupon informa- 
tion that Is related to the display from which the non- 
perceptible data was captured, or programming or con- 
trol data. 

Further in accordance with the present invention, 
the capture device includes a built-in display, such as 
an LCD display, and the encoded non-perceptible data 
includes display information, so that at least portions of 
the captured data can be discerned by a user of the de- 
vice, and the user can thus be kept informed as to the 
operation of the device. The memory and the commu- 
nications circuitry in the capture device are advanta- 
geously arranged so that, In a transaction initiated by 
the device using captured informatran, a two-way diatog 
may be Initiated between the user and an agent at a re- 
mote terminal or a POS system, and Infornnation may 
be transmitted to and stored in the device for later re- 
trieval. 

Brief Description of the Drawing 

The present invention will be more fully appreciated 
by consideration of the foltowing Detailed Description, 
which should be read in light of the accompanying draw- 
ing in which: 

Fig. 1 is a schematic drawing of one embodiment 

of a system arranged in accordance with the 

present invention for wireless capture of encoded 

data transmitted with a television signal; 

Fig. 2 is a block diagram illustrating the components 

of capture device 110 of Fig. 1 ; 

Fig. 3 is a diagram illustrating the format of captured 

data; and 

Fig. 4 is a flow diagram of the process by which data 
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embedded in a subliminal visual or non-audible 
message is captured and used. 

Detailed Description 

Referring first to Fig. 1 , there Is shown a scheniatic 
drawing of one embodiment of the present Invention ar- 
ranged for the wireless capture of encoded non-percep- 
tible data from a light emitting device such as television 
101 . and subsequent use of the captured data to initiate 
a transaction which might be handled by a person at an 
agent terminal 1 20, or a person at another terminal 1 21 
that is supported by an interactive sen/ice platform 1 30. 

A television signal is received by television 101 via 
over-the-air transmission, as from transmission tower 
102 via a transmission link 103, or alternatively is sup- 
plied by a cable TV connection, or from a VCR/video- 
tape. The video portion of the television signal Is dis- 
played on the picture tube or other visual display area 
1 05 of a monitor, computer or a television 1 01 , while the 
audio portion of the television signal is played through 
or by projected from speaker 106. Associated with the 
program that is seen and/or heard by a viewer/listener 
Is non-discemlble encoded data that Is transmitted as 
part of the audio and/or video signal. For example, the 
program may be an advertisement for a service or prod- 
uct, and the indiscernible data may create a subliminal 
visual pattern on the visual display area, that when prop- 
erly received and decoded, contains Information need- 
ed to order those services or products, including prk;es, 
delivery inten^als, shipping details, coupon offers, and 
so-on. 

Encoding, transmission and decoding of non-dis- 
cernible data may be accomplished as described In a 
patent application entitled 'System and Method for En- 
coding Digital Information in a Television Signal" filed on 
behalf of T. Sizer and assigned to the same assignee as 
the present application, which co-pending application is 
incorporated herein by reference. Briefly, it was found 
using a spectrum analyzer on a typical video signal, that 
there are comparatively large frequency components at 
the line rate and at the frame rate and Its harmonics, but 
that between these frequencies, there are other fre- 
quency bands in which little information Is carried. One 
such frequency band is between 1 5 and 30 kHz. By add- 
ing a low level carrier signal or tone at a frequency in 
this band, say 25kHz, the video image is not degraded, 
but a property tuned decoder can receive and decode 
the encoded information. In this way, digital information 
can be subliminalty inserted in a video signal by adding 
to the video signal an amplitude shift keyed (ASK) or 
frequency shift keyed (FSK) carrier signal, and the dig- 
ital information can later be recovered using ASK or FSK 
decoding. Encoded data can also be inserted in a tele- 
visbn signal and recovered by a receiver that responds 
to the picture displayed on the television, in the nnanner 
described in patent 4,807,031 cited above. 

The program displayed on television 101 is sensed 



by a portable capture device 110, which is described in 
more detail in connection with Fig. 2. When the non-dis- 
cernible encoded data is part of the video signal, such 
sensing takes the form of light from display area 105 

5 being collected by an optical lens 117, while when the 
non-discernible encoded data is part of the audio signal, 
such sensing takes the form of sound from speaker 106 
being collected by a microphone (21 1 in Fig. 2). In either 
event, there is a wireless communication of the inforrra- 

10 tion from television 1 01 to capture device 110 such that 
the encoded data is "viewed" or "heard" rather than be- 
ing transmitted through a wired connection, or through 
a more conventional radio frequency communication. 
In addition to receiving non-discernible encoded da- 

^5 ta, capture device 110 has the capability of (a) storing 
the encoded data, (b) transmitting (or otherwise output- 
ting) the encoded data (or Information retrieved using 
such data) to a renrK)te kx:ation or device, and (c) dis- 
playing the encoded data to a user. The data may in- 

20 elude infomnation for initiating a transactbn or originat- 
ing a telephone call, as well as additional Information (e. 
g., price and ordering information) associated with the 
television program being displayed on television 101. 
The remote locatbn may be agent terminals 1 20 or 1 21 , 

2S interactive se wice platform 1 30, or a point of sale sys- 
tem, and the communication between the capture de- 
vice 110 and the remote locatkxi may be one-way or 
two-way. 

In the case where encoded data is to be output from 

30 capture device 110 via a telephone call, the call may be 
initiated and transacted in one of several ways, as illus- 
trated in Fig. 1 . First, the data may be transmitted over 
a wireless link to an associated telephone base station 
109 that Is, in turn, connected to a telephone line 111 

3S that Is part of a telecommunications network 1 1 5. Com- 
munication between capture device 110 and base sta- 
tion 109 may be accomplished in a variety of ways; as 
an example, the same signaling and transmissbn can 
be used as is presently used in communication between 

^ a cordless telephone handset and its associated base 
station. Second, capture device 110 may be directly 
connected to a telephone line 112 through an interface 
that includes a standard telephone jack 114. Third, cap- 
ture device 110 may include the functionality of a cellular 

45 telephone, so that the data nnay be communicated to a 
remote tocation via a wireless call placed to a mobile 
telephone switching office (MTSO) 140, which is in turn 
connected to telecommunications network 115. 

As explained more fully in connectbn with Fig. 2, 

50 capture device 110 may include a dual tone multi-fre- 
quency (DTMF) transceiver arranged to generate a se- 
ries of audible tones. Accordingly, a fourth way to initiate 
a telephone call would apply portbns of the data to the 
transmitter (tone generator) portion of the DTMF trans- 

55 ceiver. Capture device 1 1 0 may then be placed in juxta- 
position with the microphone portion of a conventional 
telephone handset, such that the audible tones gener- 
ated in capture devbe 110 are acoustically coupled 
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through the handset to receivers in telecommunications 
network 115. 

Capture device 110 can also output encoded data 
directly, without making a telephone call. This is useful 
In conjunction with a point of sale system or the like, s 
which includes a docking station or other interface to 
which the capture device may be connected, or an IR 
receiving device or a device responding to DTMR 

If an outgoing telephone call is initiated by capture 
device 110, the call can illustratively be routed through io 
telecommunications network 115 directly to an agent 
terminal 1 20, or to an interactive sen/ice platfonm 1 30 to 
which is connected another agent terminal 121. When 
a connectbn is completed to platform 130^ the call may 
be routed by a switch matrix 131 to a voice response is 
unit 132 that operates under the control of a processor 
134. Information included in the captured encoded data 
can be provided to the attendants, or processor 1 34 can 
perform a look-up and retrieval operation in database 
1 31 , using encoded data as a key, in order to effectuate 20 
a desired transaction. 

For certain transactions, infomnation connected 
with a transaction may also be transmitted from sewice 
platform 1 30 to capture device 110, either for display to 
a user, storage for later use, or output to a point of sale 2S 
system or device. This information may, for example, be 
confirmation information, indicating that a transaction 
has been effected, or details regarding a just completed 
transaction, such as ticket infonmation relating to a travel 
reservation. 30 

Referring now to Fig. 2, there is shown a block dia- 
gram illustrating in more detail the components in one 
embodiment of capture device 110 of Fig. 1. If the en- 
coded non-perceptible data is inserted in the video por- 
tion of a television signal, the images displayed on visual 3S 
display area 105 are captured by optical lens 117 and 
an associated photodetector 203, which is arranged to 
supply an electrical signal to filter/receiver 215 repre- 
senting the image. On the other hand, if the encoded 
non -perceptible data is inserted in the audio portion of 40 
a television signal, the output from speaker 106 is cap- 
tured by a mfcrophone 211 and an associated amplifier 
213, which is arranged to supply an electrical signal to 
filter/receiver 215 representing the sound energy. In ei- 
ther event, the encoded non-perceptible data is decod- 4S 
ed in filter/receiver 215 in a manner consistent with the 
manner in which the original data was encoded. Thus, 
filter/receiver 215 can be arranged to perform the same 
functionality as the elements illustrated in Fig. 4 of the 
above referenced Broughton at al. patent, or. alterna- so 
tively, the operation of filter/receiver 215 can be as de- 
scribed in conjunction with Figs. 4 or 7 in copending ap- 
plication of TSizer. In each instance, it is noted that cap- 
ture device 110 receives and recovers encoded data by 
"watching" or 'listening to" the output of televisbn 101 ss 
(or another similar display devtee), and that the capture 
device 110 is not connectedtoteleviswn 101 or the other 
display device by a cable or wires. 



The output of filter/receiver 215, representing the 
data captured by capture device 110. is applied to a mi- 
croprocessor 220, which is arranged to perform varbus 
data processing and control functions in coordination 
with programs stored in an associated memory 221 as 
well as inputs received from a user of capture device 
110. Generally speaking, capture device 110 may oper- 
ate in an active or a passive mode. In the active mode, 
capture devkie 110 responds only to user initiated cap- 
ture commands. In the passive mode, capture device 
1 1 0 captures all of the data sent to it while in that mode. 
In either mode, captured data may be routed by micro- 
processor to memory 221 and stored for use at a later 
time, as determined by the user. The captured data may 
also be displayed on a display 225, such as a Iquid crys- 
tal display, so that a user will be aware of exactly what 
informatbn was captured. 

TTie control programs contained in memory 221 can 
implement numerous functions, which, broadly speak- 
ing, can be categorized as (a) f unctrans relating to data 
capture, such as SAVE and AUTOSAVE functions, (b) 
functions relating to output of already captured data, 
such as DIAL and SEND DATA functk>ns, (c) functions 
relating to memory manipulatbn, such as the RECALL 
and DELETE functbns. The above enumerated func- 
tions are initiated when respective buttons 241 - 246 are 
activated, and the activatk>n signal output from a button 
is applied to microprocessor 220 via a button interface 
240. Each of the functions is discussed bek)w. 

When the SAVE function is initiated by activatbn of 
button 241. capture device 110 is in the active mode, 
and the encoded data contained in the current audio or 
video output of television 101 is captured. This is ac- 
complished, for example, by activating optbal lens 117 
and photo detector 203, or microphone 211 and ampli- 
fier 21 3, for a relatively brief period of time, typically on 
the order of several seconds, so that encoded dgta as- 
sociated with the current program being displayed on 
display area 1 05 or played through the toudspeaker 1 06 
is processed by filter/receiver 21 5 and stored in menrwry 
221 . A visual or auditory cue may be contained in the 
vkJeo or audio program, to indicate to a user of a capture 
device that there is data to be captured. When the SAVE 
function is activated, mbroprocessor 220 may be ar- 
ranged to control display 225 so as to indbate the cur- 
rent status, e.g., that the capture device is in the process 
of receiving and capturing data. When the data has been 
captured, microprocessor 220 may be an-anged to con- 
trol display 225 so as to actually display some or all of 
the data, so that the user is now aware of infomnation 
that had theretofor been subliminal and indiscernible. 
The display can also indicate that the device is "aimed" 
correctly. 

When the AUTOSAVE function is initiated by acti- 
vation of button 244, capture device 1 1 0 is in the passive 
irode. In this mode, capture device 110 captures a se- 
ries of encoded data that may be contained in each of 
a series of programs that can be viewed/heard on tele- 
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vision 101. This is accomplished, for example, by acti- 
vating optical lens 117 and photo detector 203. or mi- 
crophone 21 1 and amplifier 21 3, for a relatively long pe- 
riod of time, typically on the order of several minutes or 
hours. Encoded data captured from the various pro- s 
grams being displayed on display area 1 05 or played 
through loudspeaker 106 over a long period of time are 
processed by fitter/receiver 215 and stored in memory 
221 . As with the SAVE function, when the AUTOSAVE 
function is activated, microprocessor 220 may be ar- io 
ranged to control display 225 so as to indicate the cur- 
rent status, e.g., that the capture device is in the process 
of receiving and capturing data. 

When a user of capture device 110 desires to use 
any of the data stored in memory 221 , including data is 
stored a result of the AUTOSAVE function, the contents 
of memory 221 may be retrieved and displayed on dis- 
play 225 by initiating the RECALL function upon activat- 
ing button 245, This permits a user to be aware of ex- 
actly what information was previously captured and 20 
stored. Through this function, the user will be able to 
selectively peruse or scroll through the stored data, so 
as to retain or use selected portions thereof. Activation 
of button 246 initiates the DELETE function, whereby 
portions of the data stored in memory 221 are selectively 2S 
erased. In order to consen/e and thereby 'recycle' to 
storage capacity of memory 221 , microprocessor 220 
may also be arranged to manage the contents in mem- 
ory 221, so that the oldest information stored therein 
could be cycled out and deleted as new information is 30 
captured and stored. 

Turning now to consideration of the use of already 
captured data, it is an Important functionality of capture 
device 1 1 0 to permit such data to be output upon a com- 
mand from a user. Such data, when output, can be used 3S 
to (a) nrake a telephone call, and, when the call is an- 
swered, to provkie infonmation to a live or automated 
attendant that receives the call, or (b) Initiate a transac- 
tion, such as by interacting with a special purpose ter- 
minal or kiosk arranged to receive the stored data, or a 40 
device such as a VCR Plus or a stereo. In accordance 
with the arrangement shown in Fig. 2. data output via a 
telephone call can be initiated In several ways. 

First, captured data can be routed by microproces- 
sor 220 from memory 221 to a modem 246 and then to 4S 
a base statran interface 250, so that the informatbn can 
be communicated to telephone base 109 of Fig. 1 . This 
in turn allows a conventional telephone call to be origi- 
nated from telephone base 109. 

Second, captured data can be routed through nrro- so 
dem 246 to a wire interface 253, which allows capture 
device 110 to be physrcally connected to a terminal or 
docking statbn for information off loading. 

Third, captured data can be routed to a DTMF trans- 
ceiver 247, whk:h includes a tone generator arranged to ss 
output DTMF tones. These tones can be directly con- 
nected to a telephone line 1 1 2 via a standard connectbn 
such as RT1 1 jack 114. Altematively, capture devrce 1 1 0 



can be placed in juxtaposition with the mrcrophone in a 
telephone handset, and the audible tones can be ap- 
plied and used much like an audible frequency autodi- 
ater. 

Fourth, captured data can be routed through nrx>- 
dem 246 to a cellular telephone 252, whrch can Initiate 
a cellular call. When a cellular connection Is completed, 
the captured data can be output in encoded form or as 
audible tones. 

For each of the arrangements just described, a tel- 
ephone call nnay be originated using the DIAL function. 
This function is initiated by activation of button 242. With 
this functbn, dialing or routing information, which is part 
of the data captured by capture devtee 110 and now 
stored in memory 221, is retrieved. This Information is 
then further processed, so that a telephone call can be 
initiated. In some instances, a call is originated by ap- 
plicatkxi of stored data to DTMF transceiver 247, so that 
audible tones can be output. In other instances, a data 
call can be initiated by appropriately formatting stored 
data. In yet other instances, such as when a cellular call 
is dialed, stored dialing informatkxi may be applied to 
an outgoing number register. The exact manner in which 
Information is output from capture devce 110 can vary, 
in accordance with several embodiments of the present 
invention. 

While the DIAL functbn is used to output certain 
captured information needed to initiate a telephone call 
(or other data output function), the SEND DATA function, 
initiated by activation of button 243, is used to output 
other portions of the captured data stored in memory 
221 . pertaining, for example, to a transaction that the 
user desires to make. This additional data is retrieved 
from memory 221 and appropriately transmitted, de- 
pending upon the manner and type of telephone call in- 
volved. Thus, data may be output either via base statkxi 
Interface 250 to telephone base 109, or via RJ 11 jack 
114 onto telephone line 112, or via cellular phone 252 
to MTSO 1 40. Note that if output occurs directly, without 
a telephone call, the SEND DATA functkxi may cause 
informatran from memory 221 to be output via wire In- 
terface 253. During the SEND DATA function, the infor- 
matbn being output may also be displayed on display 
225. This allows a person using the capture device to 
identify the Informatkxi that was captured, and use the 
informatbn in an actual transaction. 

It is to be noted here that capture device 1 1 0 of Fig. 
2 may optionally include an automatic speech recogni- 
tion circuit 245, an infrared (IR) transmitter 235, a hand- 
set 230 and a keypad 247. The purpose of I R transmitter 
235 Is to allow captured data to be output directly to a 
device such as a point of sale system or a VCR or the 
like, via Infrared transmlsskxi. The purpose of speech 
recogn ition circuit 245 Is to convert capt u red data to spo- 
ken phrases, whch can be played to a user or to an at- 
tendant after a telephone call is Initiated. TTie purpose 
of handset 230 and keypad 247 is to altow capture de- 
vk;e 110, in addition to being used in accordance with 
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the present invention, to be used to complete conven- 
tional dialed telephone calls. 

In one application of the present invention, a tele- 
vised advertisement for a product includes additional or- 
dering or "coupon" Information embedded in a sublimi- s 
nal visual pattern within the video image displayed on 
visual display area 105. The coupon information de- 
scribes the item(s) offered for sale, the regular price, the 
coupon value, and contains additional information need- 
ed for ordering. This embedded information is transmit- io 
ted between television 101 and capture device 11 0 at a 
rate such as 60 bits per second, that is sufficient such 
that the necessary information can be transmitted in a 
very brief period of time. In the application just de- 
scribed, the embedded product information, dialing '5 
number, coupon infornnation and necessary checkbits 
together comprise approximately 2,000 bits of infornna- 
tion. so that between three and four seconds would be 
required for transmission. Once the coupon information 
is stored in capture device 110, the user of the capture 20 
device can supplement the coupon infomoation with 
user entered data which can be entered using keypad 
247. This supplemental lnformatk)n may include credit 
card or other billing information pertaining to the user, 
as an example. 2S 

As shown in Fig. 3, the informatk)n represented by 
the encoded data can be organized in several fields. Us- 
ing the coupon application as an example, a first field 
301 contains a telephone number whk:h may be used 
to originate a telephone call in order to order the product 30 
to which the coupon applies. Illustratively, this will be a 
toll-free or "800" number Other informatk>n associated 
with the coupon can include a product description in field 
303. an expiration date in fiekJ 305, the time that the 
advertisement was televised in field 307, the number of 3S 
the television channel running the ad in field 309, and a 
serial number for the ad in field 311 . kientifying either or 
both the partbular ad and its sequence in an ad cam- 
paign. It is to be noted that various other informatbn can 
also be included in the information illustrated in Fig. 3, 40 
such as keywords or attributes describing the received 
encoded data, as well as information indicating that a 
product is available in partk:ular colors or sizes. The key- 
words would be useful in retrieving from memory 221, 
information relating to specific products or advertisers. 45 

The process by whk;h informatkxi embedded in a 
subliminal visual or audible message is captured and 
used is illustrated in flow diagram form in Fig. 4. As in- 
dicated prevbusly, two general modes of operatbn are 
contemplated, namely a "passive" mode and an "active" so 
mode, both of which follow a "start" step 401 . 

In the passive mode, a user desires to operate cap- 
ture device 1 1 0 in an unattended manner, and to capture 
all of the encoded data that may be subliminally dis- 
played on a display device 105 or output from speaker ss 
1 06 over a relatively long period of time, typk:ally several 
minutes or hours, until the user, at some later time, de- 
sires to discontinue or terminate the capture. This mode 



is initiated using AUTOSAVE functionality, by activating 
button 244. When this is done, a positive response is 
obtained in step 502, so that data capture, storage and 
display, all as described below in connection with step 
507, occur. While in the passive mode, capture is temni- 
nated by subsequent activation of the same button 224. 

In the active mode, a user operates capture device 
110 in real time, typically while viewing a video display 
or listening to an audio presentation. During the display 
or presentation, the user sees or hears a corresponding 
televised advertisement or other indication that imper- 
ceptible information is available for capture. The user 
may, at this time, be instructed to activate SAVE button 
241 , or the user may know, from previous experience, 
how to use the devtee. When button 241 is activated, a 
positive result occurs in step 503, thereby capturing the 
imperceptible encoded data, displaying all or a portbn 
of the data on display 225, and storing same in merTK)ry 
221 , all in step 507. Until the SAVE button is activated, 
capture device 110 nnay be in a "wait state" 505. 

The user of capture device 110 determines that the 
appropriate data has been captured by reviewing the in- 
formation seen in display 225. After the captured data 
has been reviewed and determined to be correct, acti- 
vation of the DIAL button 242 on the portable capture 
device 1 1 0 in step 509 initiates step 51 3. by which stored 
dialing and/or routing information, illustratively in fiekJ 
401 of Fig. 4. is recalled from memory 221 . A call is then 
originated launched, such by transmitting the dialing in- 
formation from memory 221 through base station inter- 
face 250 to base station 109 and thence to telephone 
line 111 and telecommunications network 115 to inter- 
active sen^ice platform 130. The call may be a kx^al or 
long distance call, made using a "plain old telephone 
sendee" (POTS) line; alternatively, a toll-free call may 
be launched using an 800 number. Desirably, automata 
number kJentification (AN I), caller ID. and other features 
available in the telecommunk:atk>ns networic may be 
used to provide additkxial infornDatbn to the called party, 
in additfon to the captured data that is transmitted in suc- 
ceeding steps in the process. Until the DIAL button is 
activated, capture device 110 nnay be in a "wait state" 
511. 

After the call has been originated, a determination 
is made in step 515 that the call has been answered and 
that the SEND DATA button 243 has been pushed. (Until 
the SEND DATA button is activated, capture device 110 
may be in a "wait state" 517.) When a positive result 
occurs in step 515, information contained in fields 
403-411 is transmitted to the called destination in step 
51 9. As stated previously, from the point of view of the 
called party, the information received includes details 
concerning the advertised product, the identity of the 
caller, and other information associated with the item 
that was initially displayed or mentioned on the televi- 
sk)n program. Advantageously, the advertiser will know 
more about their customer than ever before. This is be- 
cause the advertiser will be able to determine from the 
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transmitted information which advertisement drew the 
business, how long it took the caller to respond to the 
ad. and the geographic location of the callers, based 
upon AN I or caller ID. It will be easy to tell which ads in 
a campaign are most successful. 

In accordance with one optional arrangement of the 
present invention, when data is captured and stored in 
step 507, the newly collected data is compared with pre- 
viously stored data. When a match is found, any dupli- 
cate data is dropped. In addition, the infonration in var- 
ious fields is examined, to assure that the data captured 
is in the appropriate format. If an error is detected in any 
data, the infornnation may be deleted, and "new" data 
collected. This is accomplished by repeating capture 
step 507 several times. 

In another application of the present invention, an 
interactive television capability is provided in connection 
with a game show or television drama. For a game show, 
an audience would be asked to vote on various aspects 
of the game. This could include questions such as which 
person should goon a date with the contestant, or which 
singer is the best? The audience would be instructed to 
press the "SAVE" button on their capture device at the 
appropriate time specified for their choice. The viewers 
vote would then be cast by depressing the DIAL button, 
communicating with the game show host, and then 
transmitting infomnatbn indicating the vote by depress- 
ing the SEND DATA button. Alternately, one number 
could be transmitted and an automated interactive voice 
platfomri such as the InfoWonc^ platfonm available from 
AT&T could offer options to the caller These calls could 
be toll free calls, or, altematively, billed to the caller. 

In yet another application of the present invention, 
a video on demand capability can be provided. A viewer 
sees a video he or she would like to see advertised on 
a previewing network. An instruction to press the DIAL 
button on the capture device is made on the previewing 
channel to indicate to the viewer that he or she can press 
the button to order a particular video. The call wouki be 
placed to a voice response system used to reserve the 
video on demand. Information about which video the 
caller wanted to see is sent over the phone lines once 
the call is connected. The caller would merely have to 
acknowledge the order and select a time slot. 

The present invention can also be employed in the 
context of programming of a VCR, particularly where the 
user already has the capabilities provided by VCR Plus, 
a product that is commercially available. In this embod- 
iment, a code sequence associated with and represent- 
ing a particular television show constitutes the informa- 
tion included in a televised advertisement. When the ad 
is viewed, the viewer can press the SAVE button and 
send the sequences to the VCR Plus using IR transmit- 
ter 235. 

While the portability of capture device 11 0 is advan- 
tageous, it is to be noted that in some arrangements, 
the functions of capture device 110 and telephone base 
109 may be combined in a single stationary unit, which 



might resemble a cordless telephone base station. In 
this event, the functionality of capture device 110 would 
be included in the stationary unit, and that unit would be 
arranged, in the manner previously described, to collect 

5 informatbn from the visual display area 1 05 and to pro- 
vide storage of the collected information. In this arrange- 
ment, the stationary unit could also be adapted to utilize 
input devices such as optical character recognition 
(OCR) or scanners that would provide an additional 

10 method of inputting telephone numbers and coupon 
and/or product information. The hand piece of this sta- 
tionary unit would be used to place a call and send the 
data. 

The present invention is not limited to live televised 

15 programs, but can also be operated in conjunctk>n with 
stored video programs played from a VCR. In this ar- 
rangement, advertisements and other encoded informa- 
tion is included in videos at the time they are recorded 
or at the time they are mastered and put onto vkieo cas- 

20 settes. A viewer watching a video with such an ad can 
respond immediately to reserve another video, pur- 
chase one of the items advertised, or order catabgs au- 
tomatically. When a program containing subliminally en- 
coded data Is viewed, the viewer simply presses the 

25 DIAL button, launching a telephone call, and then press- 
es the SEND DATA button, sending captured data, such 
as product information, to an agent, who can verify the 
order using the additional infomnatbn that was captured 
or stored in the device. 

30 The portability of the present invention can be used 
to advantage in implementing what can, in effect, be 
called a "Roaming Bulletin Board". A user can be attend- 
ing a conference or traveling through the airport where 
kk>sks or bulletin boards are set up. In the case of a con- 

35 ference, an attendee would walkthrough the conference 
viewing the displays. When the attendee sees a display 
of interest, he or she points the capture device towards 
the display, and captures the non-discernible encoded 
data from either the displayed image or the sound out- 

^ put. Later, the user launches a telephone call to the tel- 
ephone number specified and included in the captured 
data, and sends out coupon information that is also part 
of the captured data. 

It is further to be noted that the present invention 

45 can be used in the context of automobiles and mobile 
telephones. When a capture device in accordance with 
the inventbn is placed in a car, it can receive and cap- 
ture non-discernible encoded data contained in the au- 
db output of the car radio. The captured data can in- 

50 dude a toll free or conventional telephone number for 
originating a mobile cellular telephone call from the car. 
The call may be placed to an interactive service plat- 
form, and additional captured data, such as coupon or 
similar information, can then be used to effectuate a 

55 transaction. 

Various modifications and adaptations of the 
present invention will be apparent to those skilled in the 
art. For example, the functions provided by the present 



7 



13 



EP 0 713 335 A2 



14 



invention may be connbined with those used in a per- 
sonal digital assistant (PDA) or a laptop computer. If de- 
sired, a simultaneous voice and data (SVD) modem of 
the type available now from AT&T could be provided in 
capture device 110 to allow communication both by 
voice and data transfer, at the same time. Various func- 
tions can also be added to the processes and applica- 
tions described. For example, telephone base 109 may 
be programmed to dial a number or series of numbers 
at specific times, when certain amounts of information 
or data have been collected, or based upon some crite- 
ria. Capture device 110 could then send collected infor- 
mation "in bulk' to interactive sen/ice platform 130. For 
this purpose, modem 246 could be used, so as to send 
the information more efficiently. 



Claims 

1. Apparatus for initiating a transaction, comprising 

means arranged for wireless capture of data 
directly from an audio/visual display, said data 
being indiscernible by a viewer or listener, and 
means an-anged to transfer at least portions of 
said data to a separate transaction processor 
to initiate said transaction, 

2. The invention defined in claim 1 wherein said data 
includes dialing inforrmtion for originating a tele- 
phone call, and wherein said transfer means 
includes means for initiating said telephone call 
using said dialing information. 

3. The invention defined in claim 1 wherein said appa- 
ratus further includes 

means responsive to said capture means for 
storing said data, and 

wherein said transfer means includes means 
for coupling information from said storing 
means directly to a separate transaction proc- 
essor. 

4. The invention defined in claim 1 wherein said cap- 
ture means is in a portable capture device and said 
transfer means is in a base station, and wherein 
said apparatus further includes means for transfer- 
ring information from said portable capture device 
to said base station. 

5. The invention defined in claim 1 wherein said 
audio/visual display includes non-perceptible 
encoded data, and wherein said capture means is 
arranged to receive and decode said encoded data. 

6. The invention defined in claim 5 wherein said non- 
perceptible encoded data includes an amplitude 



shift keyed (ASK) or frequency shift keyed (FSK) 
carrier signal added to a television signal, and 
wherein capture is effected by detecting and decod- 
ing said carrier signal. 

5 

7. A system for capturing data comprising 

means for wireless capture of indiscernible 
data embedded in an audkWideo display; and 
10 means for outputting at least portions of said 

captured indiscernible data to a remote device 
via a telephone call. 

8. Apparatus for initiating a transaction relating to a 
15 program appearing on a television display, said 

apparatus comprising 

means arranged for wireless capture of data 
directly from said television display, said data 
20 being indiscernible to a viewer of said televisbn 

display; 

means responsive to said captured data for 
obtaining a telephone number, said telephone 
number being associated with a remote loca- 
ls tion arranged to perfonm transactions associ- 
ated with said program appearing on said tele- 
vision display; 

means for initiating a telephone call using said 
telephone number; and 
30 means for transmitting to said remote location, 

during said telephone call, additional informa- 
tion contained in said captured data, said addi- 
tional infonmatbn also pertaining to said pro- 
gram. 

35 

9. A system for responding to a televised advertise- 
ment, comprising 

means for embedding non-discernible informa- 
40 tion in said advertisement, said information 

including (a) a telephone number for a remote 
processor arranged to process orders for the 
items displayed in said advertisement, and (b) 
other information relating to said itenns dis- 
^ played in said advertisement; 

means arranged for wireless capture of said 
non-discernible information directly from said 
televised advertisement; and 
means responsive to said captured information 
^ for (a) placing a telephone call using said tele- 

phone number and (b) transmitting said other 
infonmatbn to sakJ remote processor via said 
telephone call. 

55 1 0. A method for initiating a transaction, comprising the 
steps of 

capturing data directly from an audio/visual dis- 
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play, said data being indiscernible by a viewer 
or listener, and 

transferring at least portions of said data to a 
separate transaction processor to initiate said 
transaction. s 

11. The method defined in claim 10 wherein said data 
includes dialing information for originating a tele- 
phone call, and wherein said transferring step 
includes initiating said telephone call using said io 
dialing infornnation. 

12. A method for capturing data comprising 

capturing, using wireless capture, indiscernible 
data embedded in an audio/video display; and 
outputting at least portions of said captured 
indiscernible data to a remote device via a tel- 
ephone call. 

1 3. A method for in itiat ing a transaction relating to a pro- 
gram appearing on a television display, said method 
comprising steps of 

using wireless capture data directly from said 
television display, said data being indiscernible 
to a viewer of said television display; 
responsive to said captured data, obtaining a 
telephone number, said telephone number 
being associated with a rerriote location 
arranged to perform transact bns associated 
with said program appearing on said television 
display; 

initiating a telephone call using said telephone 
numben and 

transmitting to said remote location, during said 
telephone call, additional information con- 
tained in said captured data, said additional 
information also pertaining to said program. 

14. A method for responding to a televised advertise- 
ment, comprising the steps of 

embedding non-discernible information in said 
advertisement, said information including (a) a ^ 
telephone number for a remote processor 
arranged to process orders for the items dis- 
played in said advertisement, and (b) other 
information relating to said items displayed in 
said advertisement; 50 
capturing, using wireless capture, said non-dis- 
cernible information directly from said televised 
advertisement; and 

responsive to said captured information, (a) 
placing a telephone call using said telephone ^ 
number and (b) transmitting said other informa- 
tion to said remote processor via said tele- 
phone call. 
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